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JLt  Introduction  and  background 

The  work  described  In  this  report  Is  a  continuation  of  research 
performed  during  1990  under  the  AFOSR  Research  Initiation  Program.  That 
project  focused  on  the  development  of  a  blackboard  expert  system  which 
Integrated  several  tec^'nlques  In  the  task  of  landmark  recognition  of 
human  subjects.  It  was  able  to  successfully  locate  eleven  landmarks  on 
a  small  sample  of  subjects. 

Although  this  system  demonstrated  the  feasibility  of  this  approach 
to  the  problem,  It  was  a  first  attempt  to  build  a  program  of  this  type 
and,  as  such,  needed  major  reworking  to  support  easy  modification  and 
extension.  The  overall  design  and  knowledge  sources  were  generally 
good,  but  the  Implementation  needed  a  greater  degree  of  modularity  and 
generality. 


The  design  of  the  system  Is  described  In  the  final  report  dated 
15  September  1990  and  entitled  "A  Blackboard  Architecture  for  Landmark 
Identification  on  3-01mensional  Surface  Images  of  Human  Subjects." 
Changes  made  to  that  design  In  the  current  work  will  be  described 
below. 

Section  II  of  this  report  is  a  general  description  of  the  tasks 
attempted  during  the  course  of  this  research.  In  this  section, 
implementation  details  are  kept  to  a  minimum.  Section  III  Includes  a 
more  technical  discussion  of  design  and  Implementation  Issues.  Section 
IV  presents  the  results  obtained  using  the  expanded  and  modified  program 
on  three  categories  of  test  subjects:  "a"  pose  males,  which  was  the 
category  of  subjects  used  as  test  data  during  the  development  of  the 
program,  "b"  pose  males  and  "a"  pose  females.  A  discussion  of  these 
results  is  also  included  In  this  section.  Section  V  discusses  possible 
directions  for  future  work. 
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lit  Tasks  attcmoted/accompl ished 

A.  Task  1  -  Restructuring  and  expansion  of  the  blackboard  architecture 

The  first  task  accomplished  in  this  research  is  a  major  revision 
and  expansion  of  the  blackboard  architecture  developed  in  1990.  Although 
the  criginal  design  and  knowledge  sources  were  successful,  as  the  the 
P''ogram  grew  in  size  and  complexity  it  became  clear  that  a  more  modular 
Implementation  using  more  features  of  object-oriented  programming  would 
allow  for  more  efficient  use  of  machine  resources.  The  original  program 
consisted  of  only  three  files.  This  led  to  two  major  problems:  (1)  Any 
changes  or  additions  to  a  file  meant  recompilation  of  at  least  1500 
lines  of  code,  which  was  very  time  consuming.  (2)  The  amount  of  memory 
used  by  the  Zortech  C++  compiler  is  directly  related  to  the  size  of  the 
file  being  compiled,  and  the  system  had  very  nearly  reached  its  limit  in 
terms  of  memory  required  for  compilation  of  these  files.  Breaking  up  the 
program  into  more  files  allows  the  compiler  to  deal  with  much  smaller 
files  and  therefore  avoids  this  memory  barrier. 

In  addition  to  dividing  the  program  into  a  large  number  of  files, 
the  code  was  rewritten  to  take  greater  advantage  of  C++  classes.  For 
example,  a  general  class  of  linked  list  is  defined  in  the  files 
"llist.hpp"  and  “llist.cpp."  Specific  varieties  of  linked  lists  are  used 
in  the  program  for  such  data  structures  as  the  Agenda,  Hypothesis  Lists, 
etc.,  and  they  differ  from  one  another  primarily  in  the  type  of  element 
contained  in  the  list.  However,  these  new  classes  are  defined  to  inherit 
all  the  functions  already  defined  for  linked  sts  in  general.  For 
example,  once  a  function  has  been  written  t  isert  an  element  into  a 
linked  list,  it  can  be  used  to  insert  into  anv  inked  list  no  matter  what 
kind  of  element  it  contains.  Therefore,  the  use  of  classes  which  inherit 
from  a  general  class  allow  for  greater  reusatility  of  code. 

Once  the  overall  framework  was  revised  in  this  manner,  previously 
written  knowledge  sources  were  incorporated  with  only  minor  revisions. 
The  original  system,  however,  had  data  structures  which  held  Information 
relevant  to  only  eleven  of  the  forty-two  head  landmarks  now  recognized. 
The  data  structures  were  expanded  to  include  Information  on  all  forty- 
two  landmarks,  and  integrating  knowledge  sources  were  added  for  all 
beyond  the  original  eleven.  The  result  is  a  complete  system  which  now 
generates  output  for  all  landmarks.  If  no  hypothesis  is  posted  about  a 
particular  landmark,  the  output  coordinates  for  that  landmark  will  be  0. 
The  process  of  adding  new  knowledge  sources  is  simple  and  is  described 
in  Section  III. 


B.  Task  2  -  New  techniques  and  knowledge  sources 

The  original  program  used  three  primary  techniques  to  locate 
landmarks:  (1)  local  minima  and  maxima  along  a  longitude,  (2)  changes  in 
slope  along  a  longitude  or  latitude,  and  (3)  statistical  information 
about  distances  between  various  landmarks.  Those  techniques  were  used 
in  new  knowledge  sources  as  well;  for  example  the  supramenton  is  a  new 
landmark  which  is  located  using  minima. 
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Two  new  techniques  have  been  added  to  the  system.  The  first  Is  a 
set  of  functions  that  attempt  to  locate  the  patches  which  have  been 
pTored  on  subjects  at  many  landmark  locations  and  which  show  up  as  voids 
(zeroes)  In  the  data.  The  critical  problem  here  is  to  distinguish  those 
voids  which  represent  patches  from  those  which  are  caused  by  other 
factors.  For  example,  the  ear  and  eye  regions  usually  contain  a  number 
of  voids  which  are  artifacts  of  the  scanning  process. 

The  first  step  In  designing  a  patch  recognizer  was  to  write  code  to 
extract  radii  from  a  particular  region  from  the  complete  data  set,  which 
Is  much  to  large  to  manipulate.  A  function  was  developed  which  Is 
given  the  region  boundaries  and  retrieves  the  radii  In  that  region.  The 
region  may  be  up  to  50  by  SO  In  size.  The  patch  recognizer  first  looks 
for  areas  of  contiguous  zeroes  In  that  region.  It  rejects  areas  that 
have  more  than  20  zeroes,  since  It  Is  unlikely  that  a  patch  will  be 
that  large.  It  returns  a  list  of  remaining  possibilities  to  the 
knowledge  source  which  requested  the  patch  location.  It  Is  then  up  to 
the  knowledge  source  to  evaluate  the  possibilities,  usually  based  on 
where  a  patch  Is  expected.  This  technique  was  used  with  great  success 
In  locating  the  glabella,  left  and  right  frontotemporal e,  zygofrontale, 
Inframalar  and  Infraorbitale.  It  will  also  be  useful  In  future  work  In 
recognizing  other  landmarks  such  as  the  left  and  right  traglon,  zyglon 
and  Infrazyglon. 

The  second  technique  added  to  the  system  Is  a  neural  network. 
Currently,  the  neural  net  Is  limited  to  only  one  knowledge  source  (KS 
70),  which  attempts  to  locate  the  right  cndocanthus.  Work  on  this 
technique  Is  Incomplete;  It  hes  been  demonstrated  that  It  functions, 
but  the  weights  used  In  the  tasting  of  the  system  were  generated  by 
training  techniques  that  need  to  be  revised  and  trained  on  a  larger 
number  of  data  sets.  Once  this  has  been  done,  the  neural  net  code  should 
be  extracted  from  KS  70  and  put  Into  a  general  function  that  can  be 
called  by  multiple  knowledge  sources. 


.C.  Task  3  -  Testing  on  "a”  pose,  "b"  pose  and  female  subjects 

The  system  was  developed  and  debugged  using  data  on  18  male 
subjects  In  the  "a"  pose.  At  the  end  of  the  contract  period,  testing 
was  done  on  10  additional  subjects  from  the  same  category  and  these 
results  are  presented  and  discussed  In  Section  IV. 

Although  the  system  was  developed  using  male  subjects  In  the  "a" 
pose,  one  of  the  tasks  accomplished  In  this  research  was  to  run  the 
system  on  other  types  of  subjects,  particularly  males  In  the  "b"  pose 
and  females  in  the  "a"  pose.  This  was  done  as  an  investigatory  measure, 
to  see  what  kind  of  results  would  be  obtained.  It  was  not  expected  that 
the  system  would  be  as  successful  on  these  subjects,  but  It  was  unclear 
whether  or  not  It  would  work  at  all  on  different  types  of  subject  files. 
The  system  was  run  on  5  subjects  In  each  of  these  categories,  and  the 
results  are  presented  In  Section  IV.  Briefly,  It  was  demonstrated  that 
some  landmarks  were  located  successfully  despite  these  differences,  but 
that  adjustments  will  have  to  be  made  In  many  knowledge  source 
parameters  when  subjects  of  different  types  are  used. 
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D.  Task  4  -  User-friendly  front  end 

Due  to  the  limited  contract  period,  this  task  was  not  attempted. 
It  Is  an  Important  task,  however,  and  should  be  part  of  future  work  done 
on  the  system. 


E.  Task  5  -  Output  format 

The  original  version  of  the  system  sent  output  to  the  screen  with 
Its  findings.  The  current  version  was  modified  to  produce  a  file  in  the 
same  format  as  the  landmark  files  produced  when  points  are  manually 
located.  The  system  uses  a  different  naming  convention  than  that  used 
for  the  landmark  files;  current  landmark  file  names  consist  of  "out"  (or 
"outf"  for  female  subjects)  followed  by  the  data  file  name.  For 
example,  the  landmark  file  corresponding  to  data  file  "s195.a”  would  be 
"out195.a".  The  blackboard  system  produces  a  file  called  “s195.out." 
This  prevents  the  original  landmark  file  from  being  overwritten. 

For  comparison  purposes,  the  system  also  produces  a  file  containing 
the  variances  between  results  In  the  manually  picked  and  system 
generated  landmarks.  These  are  found  In  a  file  with  that  same  name  as 
the  result  file,  but  with  the  extension  "cmp." 
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III.  Implpmentatlon  details 


A.  File  structure 

As  discussed  above,  the  system  has  been  divided  Into  a  total  of  Id 
files.  Successful  linking  and  compilation  of  the  system  depends  upon 
and  understanding  of  the  relationships  among  the  files.  Most  files  have 
one  or  more  "Include”  statements  at  the  top,  which  Indicate  to  the 
compiler  that  the  function  and  data  definitions  contained  In  the 
Included  file  may  be  referenced  as  though  they  were  part  of  the  file 
being  compiled. 

To  describe  the  Include  file  chain,  the  following  convention  Is  used: 
<f11e  a>  < —  <f11e  b> 

I 

I 

I 

I 

<fne  c> 

means  file  b  has  an  Include  statement  for  file  a,  and  file  c  has  an 
Include  statement  for  file  b.  The  Include  chain  for  the  program  Is  set 
up  as  follows: 


bb.hpp  < —  111st. hpp  < —  hyp.hpp  < —  agenda. hpp< —  bbdata.hpp 

I  I 

I  1 

I  I 

I  I 

111st. epp  agenda. epp 

reglon.cpp  bb.cpp 

bbdata.cpp 
hyp. epp 


globals.cpp 
ksO.cpp 
ksST.cpp 
ksTO.cpp 
Iks. epp 

It  should  also  be  noted  that  references  In  a  particular  file  to 
the  contents  of  files  that  are  not  included  may  be  done  using  "extern" 
statements  for  access  to  data  structures,  and  function  prototypes  for 
function  definitions.  These  are  used  In  C++  to  ensure  type 
compatablllty. 

General  File  Descriptions: 

agenda. hpp  -  class  definitions  for  Agenda  and  History  List  and 
associated  node  types 

agenda. epp  -  function  definitions  for  classes  defined  In  agenda. hpp 
bb.hpp  -  contains  globally  used  constants 
bb.cpp  -  contains  the  main  program  code 

bbdata.hpp  -  defines  the  class  BlackBoardOata  which  Includes  most  data 
structures  used  by  knowledge  sources 
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bbdata.cpp  -  function  definitions  used  to  manipulate  BlackBoard  data 
structures 

globals.cpp  -  a  few  data  structure  declarations  that  are  used  globally, 
and  definitions  of  some  frequently  used  functions  that  are  not 
associated  with  a  particular  class 

hyp.hpp  -  class  definitions  for  hypothesis  lists  and  associated  node 
types 

hyp.cpp  -  function  definitions  for  classes  defined  in  hyp.hpp 

iks.cpp  -  function  definitions  for  all  Integrating  knowledge  sources 

ksO.cpp  -  knowledge  source  functions  for  ksO  and  ks£1-56 

ksST.cpp  -  knowledge  source  functions  for  ks57  through  ks69 

ks70.cpp  -  knowledge  source  functions  for  ks70  through  ks86 

llist.hpp  -  class  definitions  for  a  general  linked  list  class  and 
associated  node  types;  also  Includes  definitions  of  several  specific 
linked  lists  types. 

llist.cpp  -  function  definitions  for  the  classes  defined  in  llist.hpp 

region. cpp  -  function  definitions  for  finding  and  manipulating  regions, 
including  the  patch  recognizer  code 

Besides  the  subject  data  file,  two  other  data  files  are  also  used 
by  the  system.  The  file  "trig.dat”  contains  one  line  for  each  landmark. 
On  that  line  is  a  list  of  the  knowledge  sources  which  are  triggered  when 
a  hypothesis  for  that  landmark  is  posted.  At  the  beginning  of  each 
program  run,  this  information  is  read  into  an  array  of  linked  lists 
called  Triggers,  which  is  part  of  the  BlackBoard  data  structure.  When  a 
hypothesis  is  posted,  the  linked  list  associated  with  that  landmark  is 
traversed  and  a  knowledge  source  activation  record  (KSAR)  is  added  to 
the  Agenda  for  each  knowledge  source  on  that  list.  When  a  new  knowledge 
source  is  added  to  the  system,  its  number  must  be  added  to  at  least  one 
of  these  lines  or  it  will  never  execute. 

The  file  "ks.dat”  contains  the  following  Information  each 
knowledge  source  other  than  integrating  knowledge  sources:  (1)  its 
priority  (in  the  range  0. 0-1.0),  and  (2)  three  (or  fewer)  landmarks 
whose  values  are  used  in  the  code  of  this  knowledge  source  function. 
This  information  is  read  into  the  BlackBoard  data  structure  arrays 
Priorities  and  Relatives,  respectively.  They  are  used  when  KSAR’s  are 
created;  the  Priorities  information  is  used  to  determine  the  order  in 
whish  KSAR’s  are  picked  from  the  Agenda  for  execution,  and  the  Relatives 
array  is  used  to  determine  if  a  knowledge  source  picked  off  the  Agenda 
should  be  executed  based  on  the  History  List  and  the  previous  and 
current  values  of  those  landmarks  that  are  considered  Relatives. 
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since  a"!!  IKS’s  have  a  priority  of  lO  and  should  be  executed 
whenever  they  are  put  on  the  Agenda,  there  Is  no  reed  to  Include  this 
Information  for  them  In  the  ks.dat  file. 

B.  Knowledge  sources 

Knowledge  source  numbering  Is  determined  as  follows.  KSO  Is  the 
knowledge  source  that  reads  the  binary  data  files  and  creates  min  and 
max  files.  Since  this  takes  several  minutes  to  execute,  once  those 
files  are  created,  It  Is  more  efficient  to  alter  the  main  program  so 
that  KSO  does  not  execute.  This  Is  done  by  commenting  out  the  lines  In 
the  main  program  In  which  an  activation  record  for  that  knowledge  source 
Is  placed  on  the  Agenda. 

KS1  through  KS42  are  the  Integrating  knowledge  sources  and  their 
numbers  correspond  to  the  landmarks  that  they  handle.  KS43  through  50 
are  Intentionally  omitted  to  accommcdate  the  poss'ble  Inclusion  of  new 
landmarks  In  the  future.  KS51  through  KS86  are  the  other  knowledge 
sources  that  post  hypotheses  about  landmarks,  and  their  numbering  does 
not  follow  a  pattern.  KS51  Is  Initially  placed  on  the  Agenda  In  the 
main  program,  since  Its  function  Is  to  read  the  min  and  max  files  Into 
arrays  for  use  by  other  knowledge  sources.  KS52  Is  also  placed  on  the 
Agenda;  Its  function  Is  to  post  a  hypothesis  about  the  promenton.  The 
posting  of  this  hypothesis  then  causes  the  normal  triggering  cycle  to 
begin.  The  other  knowledge  source  functions  are  documented  in  the  code. 

When  a  new  knowledge  source  Is  added  to  the  system,  the  following 
steps  must  be  taken: 

(1)  In  the  file  "bbdata.cpp"  there  Is  a  list  of  all  knowledge  source 
function  prototypes,  and  Its  prototype  must  be  added  to  that  list. 

(2)  In  the  same  file,  an  array  of  pointers  to  functions  is  initialized 
so  that  each  array  position  has  the  name  of  the  function  that  should  be 
executed  when  that  knowledge  source  is  activated.  A  line  should  be 
added  to  this  Initialization  code  which  corresponds  to  the  new  knowledge 
source. 


(3)  A  global  constant,  Numof  KSs,  found  in  the  file  bb.hpp,  should  be 
Incremented  once  for  each  new  knowle^dge  source.  Its  value  should  be  one 
higher  than  the  number  of  the  last  knowledge  source. 

I 

(4)  The  file  "ks.dat"  must  be  updated  to  Include  Information  on  the  new 

knowledge  source.  1 

1 

(5)  The  file  "trlg.dat”  must  be  u;!idated  so  that  the  posting  of  a 

hypothesis  on  at  least  one  landmark  will  trigger  the  new  knowledge 
source,  1 

C.  Patch  recognition 

One  of  the  new  techniques  used  In  the  system  Is  the  patch 
recognizer,  which  was  discussed  In  Section  II.  The  algorithms  used  In 
the  Implementation  of  this  technique  are  described  in  this  section.  The 
primary  function  responsible  for  the  process  Is  function  getvolds,  found 
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In  the  file  "region. cpp.“  Additional  code,  primarily  class  definitions, 
Is  found  In  "lllst.hpp." 

The  first  stage  In  this  process  Is  the  location  of  all  occurrences 
of  zero  In  the  region  being  Investigated.  A  linked  list  Is  constructed  In 
which  each  node  contains  the  coordinates  of  one  occurrence  of  zero  and 
the  number  of  neighboring  locations  (out  of  eight  possible)  which  also 
contain  zeroes. 

Function  markmistakes  attempts  to  locate  areas  of  contiguous 
zeroes  and  remove  them  from  consideration  If  they  are  too  large  to  be 
patches.  It  traverses  the  linked  list  created  In  stage  one.  For  each 
node  whose  coordinates  contain  a  zero  (Initially  all  nodes),  It  calls 
function  growcore  which  changes  the  zero  to  a  value  of  -100,  and  then 
marks  the  area  It  belongs  to  by  “growing"  It  from  the  original  location. 
That  Is,  It  looks  at  the  contents  of  all  locations  which  are  a  distance 
of  one  from  the  original  coordinates,  and  changes  each  zero  to  -100  If  It 
has  at  least  one  neighbor  which  Is  part  of  the  area  being  marked.  It 
then  proceeds  to  check  the  circle  of  locations  which  are  a  distance  of 
two,  and  so  on,  counting  the  number  of  zeroes  In  the  area  as  It  makes 
these  changes.  The  termination  condition  Is  a  checking  a  ccmplete  circle 
without  finding  any  zeroes.  At  that  point,  all  the  zeroes  In  the  area 
have  been  changed  to  -100's  and  we  know  how  many  of  them  there  are.  An 
area  which  Is  too  big  Is  currently  defined  to  be  one  which  has  more  than 
20  zeroes.  If  such  an  area  Is  found,  the  -100's  are  left  In  place.  If 
the  area  has  20  or  fewer  zeroes,  function  growcore  Is  called  again,  but 
this  time  It  changes  the  -100's  back  to  zeroes,  and  this  area  remains  a 
patch  candidate. 

As  function  markmistakes  traverses  the  linked  list  of  zeroes,  It 
also  destroys  It.  When  It  has  completed  Its  task,  the  Initial  process  of 
locating  zeroes  and  putting  them  Into  a  linked  list  Is  repeated; 
however,  this  time  the  list  will  be  much  smaller  because  the  large  areas 
of  zeroes  which  are  unlikely  patches  have  all  been  changed  to  -lOO's. 
This  linked  list  Is  returned  from  the  call  to  function  getvolds. 

Additional  details  may  be  found  In  the  documentation  associated 
with  these  functions. 
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Results 

Tables  1  through  4  show  the  results  obta^ned  by  the  program  on 
four  different  subject  groups.  Each  table  presents  the  variances 
between  manually  picked  and  system  generated  landmark  coordinates.  Also 
shown  for  each  coordinate  Is  the  average  variance  and  the  count  of 
system  generated  coordinates  which  are  within  3  of  the  picked  points 
(I.e.,  a  close  match),  In  the  range  4-6  from  the  picked  points  (I.e.,  In 
the  general  vicinity),  and  those  over  6  away  (I.e.,  seriously  off).  A 
failure  to  post  a  hypothesis  for  a  landmark  Is  added  to  the  count  of 
those  over  C  away.  These  counts  are  Important  because  a  serious  error  in 
the  generated  landmark  in  one  subject  can  have  a  major  impact  on  the 
average  since  the  sample  is  so  small.  It  is  more  informative  to  look  at 
the  number  of  cases  in  which  the  system  generated  the  same  (or 
different)  results  as  those  found  in  the  manually  picked  landmark  file. 

It  should  also  be  noted  that  the  manually  picked  points  are  not 
error  free.  For  example,  the  picked  longitudes  of  the  menton  and 
promenton  in  file  out06.a  differ  by  12,  which  Is  highly  unlikely. 

A  total  of  21  landmarks  are  currently  recognized  by  the  system. 
The  supramenton  is  difficult  to  evaluate  because  It  has  been  manually 
picked  on  only  three  of  the  test  subjects.  The  supramenton  has  been 
omitted  from  the  tables  for  this  reason.  In  all  three  cases,  however, 
the  generated  and  picked  points  agree  exactly,  and  inspection  of  the 
other  generated  supramentons  show  a  reasonable  position  relative  to  the 
promenton  and  stomion.  There  are  knowledge  sources  in  place  that 
attempt  to  locate  the  tragions,  which  are  not  counted  in  21  landmarks 
mentioned  above,  but  since  these  show  little  accuracy  at  the  present 
time  these  results  have  not  been  Included. 

Another  point  that  should  be  made  before  discussing  specific 
results  is  that  there  are  two  landmarks  which  the  system  initially 
locates  without  reference  to  any  other  landmarks.  The  promenton 
coordinates  are  determined  by  using  max  and  min  data,  and  the  glabella 
is  located  using  max  data  and  the  patch  recognizer.  In  all  cases,  the 
system  is  correct  on  at  least  one  of  these  two  points.  When  they  are 
inconsisteiit,  particularly  with  regard  to  longitude,  the  system  must 
choose  between  the  hypothesized  longitudes  to  use  as  a  basis  for 
locating  other  landmarks  on  the  mid-sagittal  plane.  Currently,  the 
promenton  knowledge  source  is  considered  to  be  more  reliable.  If  the 
patch  hypothesized  to  represent  the  glabella  is  greater  than  10 
longitudes  from  the  promenton  longitude,  another  patch  is  sought  whose 
longitude  Is  closer  to  that  of  the  promenton.  When  the  promenton 
longitude  is  Incorrect,  most  other  landmarks  will  be  Incorrectly  located 
as  well.  Clearly,  more  work  needs  to  be  done  to  make  this  initial 
decision  more  reliable.  Similarly,  If  the  promenton  latitude  is 
Incorrect,  the  latitudes  of  other  landmarks  will  most  likely  be  In 
error. 


Looking  at  male  subjects  in  the  "a"  pose,  the  category  for  which 
the  knowledge  sources  were  developed,  we  have  Table  1  which  Includes 
those  subjects  used  during  program  development  (hereafter  referred  to  as 
development  subjects),  and  Table  2  which  includes  new  subjects  tested 
only  after  this  stage  of  program  development  was  completed  (test 
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nf  =  not  found  by  the  system  np  =  not  manually  picked 

c<4  =  count  of  variances  less  than  4  for  each  landmark  coordinate 
c4-6  =  count  of  variances  from  4  to  6  for  each  landmark  coordinate 
c>6  =  count  of  variances  greater  than  6  for  each  landmark  coordinate 
avg*  :  average  variances,  excluding  data  on  subjects  s04S  and  si 37 
avg  s  overall  average  variances 


Table  1.  Variances  between  generated  and  manually  picked  coordinates  for 
male  subjects  In  "a"  pose  (development  group),  (continued  on  next  page) 


nf  =  not  found  by  the  system  np  =  not  manually  picked 

c<4  =  count  of  variances  less  than  4  for  each  landmark  coordinate 
c4-6  =  count  of  variances  from  4  to  6  for  each  landmark  coordinate 
c>6  =  count  of  variances  greater  than  6  for  each  landmark  coordinate 
avg*  =  average  variances,  excluding  data  on  subjects  s045  and  s137 
avg  s  overall  average  variances 

Table  1.  (cont’d)  Variances  between  generated  and  manually  picked 
coordinates  for  male  subjects  In  "a"  pose  (development  group). 
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nf  =  not  found  by  the  system  np  =  not  manually  picked 

c<4  =  count  of  variances  less  than  4  for  each  landmark  coordinate 
c4-6  =  count  of  variances  from  4  to  6  for  each  landmark  coordinate 
06  =  count  of  variances  greater  than  6  for  each  landmark  coordinate 
avg«  =  average  variances,  excluding  data  on  subjects  s109  and  s13S 
avg  =  overall  average  variances 


Table  2.  Variances  between  generated  and  manually  picked  coordinates  for 
male  subjects  in  "a”  pose  (test  group). 


nf  =  not  found  by  the  system  np  =  not  manually  picked 

c<4  =  count  of  variances  less  than  4  for  each  landmark  coordinate 
c4-6  =  count  of  variances  from  4  to  6  for  each  landmark  coordinate 
c>6  =  count  of  variances  greater  than  6  for  each  landmark  coordinate 
avg  =  overall  average  variances 

Table  3.  Variances  between  generated  and  manually  picked  coordinates  for 
male  subjects  In  “b"  pose. 
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nf  =  not  found  by  the  system  np  =  not  manually  picked 

c<4  =  count  of  variances  less  than  4  for  each  landmark  coordinate 
C4-6  =  count  of  variances  from  4  to  6  for  each  landmark  coordinate 
c>6  =  count  of  variances  greater  than  6  for  each  landmark  coordinate 
avg  =  overall  average  variances 


Table  4.  Variances  between  generated  and  manually  picked  coordinates  for 
female  subjects  In  “a"  pose 


to  ICO  1(0  ICO 


subjects).  Table  1  shows  a  greater  degree  of  accuracy  than  Table  2, 
which  is  to  be  expected.  In  Table  1,  it  can  be  seen  that  the  results  of 
only  2  subjects  of  18  are  largely  Incorrect  (s045  and  s137).  In  Table 
2,  2  of  10  (s109  and  s135)  are  largely  incorrect.  These  subjects  were 
omitted  from  the  averages  shown  in  the  columns  labelled  "avg*”  but  were 
included  in  all  counts  and  the  overall  average  (column  “avg**). 

Table  1  (development  subjects)  shows  that  when  the  promenton  is 
correctly  identified,  there  are  only  9  coordinates  showing  an  average 
variance  of  4  or  greater,  out  of  a  total  of  40  coordinates.  Of  this 
group  of  9,  only  1  shows  an  average  variance  greater  than  6.  For  the 
test  group  (Table  2),  however,  20  out  of  40  coordinates  show  a  variance 
of  4  or  more,  even  when  the  promenton  is  correctly  identified. 

Counts  tell  a  slightly  different  story.  Keeping  in  mind  that  2  of 
the  10  datasets  in  the  test  group  have  an  incorrect  promenton,  which 
affects  most  other  landmarks,  20  coordinates  out  of  40  are  found  with 
high  accuracy  (variance  of  3  or  less)  in  at  least  70X  of  the  subjects. 
Generated  coordinates  are  in  the  vicinity  (variance  within  6)  of  picked 
coordinates  in  at  least  70X  of  the  subjects  for  28  out  of  40  points. 
The  only  coordinate  which  was  completely  missed  (variance  over  6)  in  the 
majority  of  subjects  is  the  right  endocanthus  longitude. 

Table  3  shows  the  results  for  5  male  subjects  in  the  "b"  pose. 
There  are  some  interesting  observations  that  can  be  made.  First,  the 
promenton  longitude  (and  therefore  the  mid-sagittal  plane)  was 
successfully  located  in  all  subjects,  with  variances  ranging  from  1  to 
5.  The  promenton  latitude,  however,  was  seriously  off  in  3  of  the  5 
cases,  leading  to  errors  in  the  latitudes  hypothesized  for  most  of  the 
other  landmarks.  Despite  this,  there  were  three  landmarks  that  showed 
perfect  matches  between  generated  and  picked  points:  the  right  and  left 
inframalar  and  the  right  infraorbitale,  all  of  which  are  located  using 
the  patch  recognizer.  The  pronasale  and  subnasale  were  located  with 
good  accuracy  as  well.  It  is  likely  that  modifications  to  knowledge 
source  parameters  will  allow  the  system  to  generate  results  on  "b"  pose 
males  that  are  comparable  to  "a"  pose  males. 

Table  4  shows  the  results  for  5  female  subjects  in  the  “a”  pose. 
Subject  flO  shows  remarkably  good  results,  comparable  to  the  best  male 
“a"  pose  subjects.  In  the  others,  the  promenton  longitude  is  found  with 
good  accuracy,  but  there  are  major  errors  in  latitude,  as  was  the  case 
with  "b"  pose  males.  This  is  apparently  an  important  area  to 
investigate.  Unlike  the  "b”  pose  males,  the  landmarks  located  using  the 
patch  recognizer  were  not  found  very  well,  but  it  may  also  be  the  case 
here  that  alterations  in  knowledge  source  parameters  will  produce  more 
acceptable  results. 


Vj.  Future  Work 


There  are  several  directions  in  which  work  should  proceed  on  this 
system.  Many  of  these  have  been  mentioned  in  other  sections  of  this 
report,  but  they  are  summarized  below. 


A.  Neural  network  completion 

The  training  of  neural  networks  to  aid  in  the  recognition  of 
landmarks  should  be  continued.  If  this  technique  is  successful  on  the 
test  case  (right  endocanthus),  other  areas  in  which  it  may  be  useful 
should  be  investigated.  The  eye  area  landmarks  are  clearly  in  that 
category,  and  the  chelions  may  also  bo  found  in  this  way  with  greater 
accuracy  than  the  system  currently  exhibits. 


B.  Development  of  integrating  knowledge  sources 

There  are  many  Instances  where  the  system  has  a  correct  hypothesis 
for  a  landmark’s  location,  but  chooses  an  incorrect  hypothesis  instead. 
This  is  the  work  of  the  integrating  knowledge  sources,  which  should  be 
the  “most  intelligent"  part  of  the  system.  This  area  is  one  which 
definitely  needs  further  development.  More  heuristics  must  be  built 
into  this  part  of  the  system  to  improve  the  decision-making  ability  o1‘ 
the  IKS’s. 

The  incorporation  of  more  knowledge  sources  with  information  about 
new  landmarks  should  Improve  system  performance  as  well,  especially  if 
more  landmarks  can  be  located  by  manipulating  the  raw  data  rather  than 
using  previously  identified  landmarks  as  a  reference.  These 
"independent"  landmark  locations  can  then  be  compared  more  easily  to  one 
another  to  check  for  consistency.  This  is  currently  the  case  only  with 
the  glabella  and  promenton,  so  it  is  difficult  for  the  system  to  choose 
the  correct  one  when  they  are  not  consistent. 


C.  User  Interface 

There  are  several  ways  in  which  user  interaction  may  be  added  to 
the  current  system.  If  a  graphics  module  is  added  to  the  system  so  that 
an  image  of  the  subject  is  displayed  with  hypothesized  landmarks,  it 
will  be  possible  for  the  user  to  have  some  input  in  the  recognition 
process.  For  example,  if  the  hypothesized  locations  of  the  glabella 
and  promenton  are  inconsistent,  as  discussed  above,  it  would  be  possible 
for  the  system  to  allow  the  user  to  make  the  decision  as  to  which  is 
more  correct. 
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